Skip to content

feat(js/plugins/vercel-ai-sdk): add @genkit-ai/vercel-ai-sdk plugin#5091

Open
chrisraygill wants to merge 16 commits intomainfrom
feat/js-vercel-ai-sdk-plugin
Open

feat(js/plugins/vercel-ai-sdk): add @genkit-ai/vercel-ai-sdk plugin#5091
chrisraygill wants to merge 16 commits intomainfrom
feat/js-vercel-ai-sdk-plugin

Conversation

@chrisraygill
Copy link
Copy Markdown
Contributor

Adds a new plugin that bridges Genkit streaming flows with the Vercel AI SDK UI hooks (useChat, useCompletion, useObject).

Plugin features:

  • chatHandler(): wraps a Genkit flow to serve the useChat() SSE protocol, supporting text, reasoning, tool request/result, file output, source citations, custom data chunks, and step markers
  • completionHandler(): wraps a Genkit flow for useCompletion(), with both SSE (data) and plain text stream protocol modes
  • objectHandler(): wraps a Genkit flow for useObject() structured output
  • AiSdkChunkSchema: typed discriminated union for flow streamSchema
  • MessagesSchema: input schema for chat flows with UIMessage conversion
  • Full contextProvider and abortSignal support across all three handlers
  • Uses createUIMessageStream / createUIMessageStreamResponse from the ai package directly, ensuring the SSE wire format stays in sync with the SDK

Also adds a Next.js testapp (js/testapps/vercel-ai-sdk-next) demonstrating all three handlers with Gemini via @genkit-ai/google-genai.

Description here... Help the reviewer by:

  • linking to an issue that includes more details
  • if it's a new feature include samples of how to use the new feature
  • (optional if issue link is provided) if you fixed a bug include basic bug details

Checklist (if applicable):

Adds a new plugin that bridges Genkit streaming flows with the Vercel AI
SDK UI hooks (useChat, useCompletion, useObject).

Plugin features:
- chatHandler(): wraps a Genkit flow to serve the useChat() SSE protocol,
  supporting text, reasoning, tool request/result, file output, source
  citations, custom data chunks, and step markers
- completionHandler(): wraps a Genkit flow for useCompletion(), with both
  SSE (data) and plain text stream protocol modes
- objectHandler(): wraps a Genkit flow for useObject() structured output
- AiSdkChunkSchema: typed discriminated union for flow streamSchema
- MessagesSchema: input schema for chat flows with UIMessage conversion
- Full contextProvider and abortSignal support across all three handlers
- Uses createUIMessageStream / createUIMessageStreamResponse from the ai
  package directly, ensuring the SSE wire format stays in sync with the SDK

Also adds a Next.js testapp (js/testapps/vercel-ai-sdk-next) demonstrating
all three handlers with Gemini via @genkit-ai/google-genai.
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new package @genkit-ai/vercel-ai-sdk to provide adapters for Vercel AI SDK UI hooks (useChat, useCompletion, useObject), enabling Genkit flows to function as backends for these hooks. The implementation includes handlers for these hooks, conversion utilities for message formats, and chunk dispatching logic. I have identified a few issues: the custom data type dispatching in dispatch.ts will cause data to be ignored by standard hooks, the use of globalThis.crypto.randomUUID() may cause compatibility issues with older Node.js versions, and the error handling in objectHandler will produce invalid JSON streams.

…mid-stream error

Appending a JSON error object to a partial JSON stream corrupts it
(e.g. {"foo":"ba{"error":"..."}) causing useObject to fail to parse.
Close the stream immediately instead, which triggers useObject's onError
callback on the client.

Also changes onError signature to (err: unknown) => void since the return
value was never meaningful in this context.
@chrisraygill chrisraygill requested a review from pavelgj April 10, 2026 03:48
…ort paths

NotificationsSchema and Notification type were imported but the file was never
created. Placed at src/schemas.ts (not src/lib/schemas.ts) to avoid the root
.gitignore rule that ignores any directory named 'lib'.
AiSdkChunkSchema → StreamChunkSchema (removes redundant vendor prefix)
ChatFlowOutputSchema → FlowOutputSchema (applies to both chat and completion)

Also renames the corresponding TypeScript types:
  AiSdkChunk → StreamChunk
  ChatFlowOutput → FlowOutput
@github-actions github-actions bot added the docs Improvements or additions to documentation label Apr 10, 2026
chrisraygill and others added 9 commits April 10, 2026 10:42
Converts a Genkit GenerateResponseChunk to StreamChunk[] values ready to
pass to sendChunk(), eliminating boilerplate in flow implementations.

Handles text deltas, reasoning/thinking deltas, and tool-request parts.
Falls back to toolRequest.name when ref is not set.

Also uses toStreamChunks() in the vercel-ai-sdk-next testapp chat flow.
…elpers

toStreamChunks converts a GenerateResponseChunk into StreamChunk[] values
ready to pass to sendChunk(), handling text, reasoning, media parts, and
tool requests in the correct emission order.

toFlowOutput extracts finishReason and usage from a GenerateResponse,
eliminating the manual return boilerplate in flow definitions.

Update the testapp chat flow to use both helpers.
…vements

- Delete unused resolveContext utility (was exported but never called)
- Remove normalizeFinishReason/FinishReason from public API (internal detail)
- Unify onError to (err: unknown) => string | void across all three handlers;
  chat/completion use the return value, objectHandler ignores it (protocol
  constraint), but the signature is now consistent
- Make ChatHandlerOptions, CompletionHandlerOptions, ObjectHandlerOptions
  generic over Ctx for typed contextProvider implementations
- Remove backward-compat plain string dispatch from dispatchChunk; flows
  must use StreamChunkSchema typed chunks (plugin is unreleased, no compat needed)
- Replace as any casts in completion.ts text mode with StreamChunkSchema.safeParse
- Add explanatory comment for unavoidable data-* cast in dispatch.ts
- Throw on legacy UIMessage role 'tool' in toGenkitMessages with a clear
  message directing users to tool-invocation parts (AI SDK v4+ format)
- Add onError and mid-stream error tests for chatHandler and completionHandler
- Add finish-reason normalisation test (unknown reasons are silently omitted)
- Add tool role throw test to convert_test.ts
- Update testapp chat flow with getWeather tool and custom data chunk;
  update chat page to render tool-invocation and reasoning parts
- Fix JSDoc in chat.ts, completion.ts, dispatch.ts that still referenced
  removed z.string() plain-string streamSchema support
- Remove z re-export from schema.ts (peer dep, consumers already have it)
- Remove ContentPartSchema/ContentPart from public API (schema internals)
- Remove closeOpenBlocks/createDispatchState/dispatchChunk/DispatchState
  from public API (implementation details; can be re-exposed if needed)
- Replace hand-rolled GenkitMessageData/GenkitPart/etc shadow types in
  convert.ts with Genkit's authoritative MessageData and Part types;
  remove GenkitMessageData and GenkitPart from public exports
- Fix ContextProvider generic order: ContextProvider<Ctx, any> not
  ContextProvider<any, Ctx> — first param is context (return type),
  second is request input type; this also eliminates the as Record<string,unknown>
  casts that were papering over the wrong order
… alignment

Switch handler flow parameters from raw Zod Action types to structural
types with named interfaces (ChatFlowInput) so TypeScript errors say
"not assignable to ChatFlowInput" instead of expanding full Zod generic
walls. Update objectHandler to use structural ObjectFlow type for
consistency with chatHandler and completionHandler.

Update vendored UIMessage types from v4/v5 (tool-invocation, args/result)
to v6 format (dynamic-tool / tool-${name}, input/output fields with new
states: input-streaming, input-available, output-available, output-error,
output-denied).

Add 4 new StreamChunkSchema variants for v6 tool lifecycle events:
tool-input-error, tool-output-error, tool-output-denied, and
tool-approval-request — with full dispatch support in stream.ts.

Replace dispatch.ts/writer.ts with stream.ts using AI SDK's own
createUIMessageStream/createUIMessageStreamResponse for SSE lifecycle.

Add compile-time type checking (tests/type_check.ts) to verify handler
parameter constraints produce errors for wrong flow types.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…losed stream

When the client aborts a request, the WritableStream may already be
closed by the time the finally block runs, causing an unhandled
ERR_INVALID_STATE rejection. Wrap writer.close() in a try/catch in
both objectHandler and completionHandler (text mode).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

config docs Improvements or additions to documentation js

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant